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ABSTRACT 

This paper explores the common elements identified as 
context variables that may effect student evaluation of instruction, 
and presents literature from sociology, anthropology, and linguistics 
that offers renewed challenges to researchers in this area of data 
interpretation* The common context variables that are seen as 
effecting student evaluation of instruction are as follows: (1) 
course variables, such as required/elective, day or evening, course 
level, lecture versus discussion, and others; (2) instructor 
variables, such as rank, gender, full-time versus part-time, years 
teaching, individual rapport, and personality characteristics; (3) 
student variables, such as academic major versus minor, full-time 
versus part-time, gender, personality characteristics, and others; 
(4) administration variables, such as student anonymity, direction 
giving, timing of semester evaluations, etc.; and (5) instrument 
variables, such as placement of items, number of response 
alternatives, negative wording of items, and the labeling of all 
scale points versus labeling only end points. The paper examines 
teaching effectiveness through the communication style used by the 
instructor, as well as the biased expectations that can be held by 
instructors of their studenti>' intellectual performance. Conversely, 
student bias towards teachers is discussed within the contexts of 
gender and sociocultural stereotyping. Contains 46 references. 
(GLR) 
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IMTERPRETATION OF STUDENT DATA: CONTEXTUAL AND 
SOCIOCULTURAL VARIABLES 

Speech communication theories share a similarity 
with the interpretation of student evaluation of 
Instruction . contextual variables. The field of speech 
communication has identified two types of theories: a 
general theory of human communication and style specific 
theory <Baker, 1984; Bormann, 1980). The general theory 
of human communication is a set of generalizations about 
communication that hold regardless of any specific 
context. The style specific theory is based on the 
assumption that communication styles may differ: among 
specific groups of people, common language usage reflects 
cultural values, different verbal and nonverbal 
communication norms, stereotyping and prejudice. 

The similarity of research is found with 
interpretation of student evaluation of instruction when 
the question is asked, "Can generalizations about teaching 
effectiveness as evaluated by student ratings of 
instruction hold regardless of any specific context 



variable?" Meta-analysis, a quantitative method used to 
convert individual study outcomes to a common 
denominator, has attempted to determine what context 
variables may effect student ratings of teacher 
effectiveness characteristics. 

The presentation today will explore the common 
elements Fdentified as context variables that may effect 
student evaluation of instruction and present literature 
from sociology, anthropology, and linguistics, that seeks 
renewed challenges to researchers in this area. These I 
have called the sociocuitur^l context variables which are: 
gender, race, ethnicity, and social class variables. The 
Sociocultural context variables will be discussed in terms 
of stereotyping and prejudice found with non-dominant 
groups. 

According to Braskamp, Brandenburg & Ory (1984) 
students are viewed as appropriate sources of describing 
and judging, such ac: 1. student-instructor relationships, 
2. their views of the instruct^- ' professional and ethical 
behavior, 3. their workload, 4. what they have learned in 
the course, 5. fairness of grading, and 6. the instructors' 
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ability to communicate cicariy. Dimensions of 
commonality (Brandenburg, Derry & Hengstler, 1978; 
Centra,1980; Doyle, 1983; Mckeachie, 1979) in 
questionnaire items have identified the following areas: 
communication skill, rapport with students, course 
organization, student self-rated accomplishments, course 
difficulty, -^and grading and examinations. 

Context variables identified with affecting student 
evaluation of mstruction are numerous and somewhat 
inconsistent. Discrepancy in conclusions according to 
Abrami, Cohen & d'Apollonia (1988) are partly attributable 
to methodology and interpretation discrepancies. The use 
of meta-analysis versus the traditional narrative review, 
has discrepancies in the interpretation by reviewers of 
collective findings. 

Common context variables as identified in the 
literature (Brady, 1989; Braskamp, et al, 1984; Cranton & 
Smith, 1986; Hoffman, 1978; Marsh, 1980, 1984; Marsh, 
Overall & Kesler, 1979) encompass the following broad 
dimensions: (See Table 1) 



course variables , such as, required/elective, day or 
evening, course level, lecture v discussion, learning value, 
organization, exams/grades, assignments, workload 
difficulty, class size, and academic discipline; 

instructor variables, such as, rank, gender, full-time 
V. part-time, individual rapport, years teaching, and 
personality " characteristics (i.e., warmth, enthusiasm); 

student variables, such as, full-time v. part-time, 
expected grade, prior interest, academic rr^ajor v. minor, 
grade point average, group interaction, gender, and 
personality characteristics; 

administration variables . such as, student anonymity, 
direction giving, instructor remains in classroom, when in 
semester evaluations are given, etc; and, 

instrument variables, such as, placement of items, 
number of response alternatives, negative wording of 
items, and the labeling of all scale points . versus labeling 
only end points. 



Cashin (1988) has statec' that the multidimensionality 
of factors when discussed in terms of reliability 
(consistency, stability and generalizability) concluded that 
no single rating Item or set of related items can be useful 
for all purposes. 

Now we will explore the communicator aspect. 
According" \o Norton (1977) teaching effectiveness is a 
function of ones' perceived communicator style. 
Communicator style is the way one verbally and 
paraverbally interacts to signal how literal meaning should 
be taken, interpreted, or understood: dominant, dramatic, 
animated, open, contentious, relaxed, friendly, tUentive, 
impression leaving, or precise. 

Babad, Bernieri, and Rosenthal (1989) found that 
students' social class and ethnicity serve as major 
variables in the formation of te??chers' expectations ,for 
students' intellectual performance. Highly biased teachers 
are those susceptible to stereotypical ly biasing 
information manifesting a strong self-fulfilling prophecy. 
In their study, biased teachers showed a highly differential, 
pattern towards either the Pygmalion effect, which is 
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positive treatment of high expectancy students, or the 
Goiom effect, which is negative treatment of low 
expectancy students. Bias towards teachers has also been 
found. 

Gender Stereotyping 

Studfnt bias regarding female faculty has been found 
to affect student evaluations of instruction (Basow & 
Distenfeld, 1985; Basow & Howe, 1987; Bennett, 1982; 
Harris, 1976; Kaschak, 1978, 1981). Researchers have 
found that depending on the methodology, the sex-typing of 
the field discussed, the sex-typed characteristics of the 
instructors, and the types of questions asked, female 
professors sometimes receive lower ratings than male 
professors, especially from male students. 

Martin (1984) felt that students have ambivalent 
expectations of female faculty. Women are supposed to be 
warm, friendly, supportive, and deferential, yet 
professionals are supposed to be objective, authoritarian, 
and critical. She believed that female faculty hsve to walk 
a fine line bfctweeo feminine warmth and masculine 
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professionalism: straying on either side might result In 
lower ratings. 

Bennett (1982) found that women are more negatively 
evaluated than men if they fall to meet gender-appropriate 
expectations with regard to student contact and support. 
Among the personal characteristics associated with high 
ratings for female but not male instructors were 
confidence and decisiveness. It seemed to Martin (1984) - 
that female faculty must try harder than their male 
colleagues to convince students that they are both well 
prepared and likeable. Similnrly, Bennetts' data (1982) 
revealed students are less tolerant of female instructors 
in a number of respects, expecting more ol them than their 
male colleagues in both educational and interpersonal 
aspects of teaching. 

Wheeless & Potori (1989), and Schein (1975) 
postulated that sex role congruency hypothesis advocates 
behavior that is consistent with sex role stereotypes 
produces more positive outcomes than behavior that is 
inconsistent. Hence, when men and wc.nen communicate 
and behave In ways congruent with societal definitions of 
dominant/masculine and expressive/feminine behaviors, 
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positive outcomes will occur. Masculine qualities of 
teaching effectiveness are decisiveness and deliberateness 
(Kearney & McCroskey, 1980) and the maintaining of control 
(Deshpande, Webb & Marks, 1970). Feminine qualities of 
teaching effectiveness are immediacy (Andersen, 1979) non 
dominant behavioi' (Norton, 1977), and caring and 
understanding (Rubin, 1981). 

A highly structured instructional approach as 
described by students as communicating greater 
professionalism was found to be consistently more 
important for females' performance ratings than for males 
(Bennett, 1982). This was especially true for students' 
ratings of instructors organization, clarity and coherence 
in classroom presentation, command of material for 
classroom presentation, and overall evaluation. Bennett 
(1982) found that students are clearly more tolerant of 
what they perceive as a lack of formal professionalism in 
the conduct of teaching from male professors, oamanding 
of women a higher staidard of formal preparation and 
organization. Research (Harris, 1976) on sex-role 
stereotyping has suggested that both maies and females 
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tenc to perceive male stereotyped behaviors as superior to 
and healthier than female-stereotyped behaviors. 

Sufficient research and documentation (Baird & 
Bradley, 1979; Kahn, 1984; Kotthoff, 1984; Shakeshaft, 
1987) has defined perceived differences and behavioral 
differences pertaining to gender patterns of 
communication. This documentation show that two distinct 
language patterns exist, one for women as^d one for men. 
(See Tables 2 and 3) Examples of these distinct 
language patterns are: 



Female Patterns 



Male Patterns 



Correct speech forms 
joking 



More frequent use of 



Dynamic intonations, wide 
range of pitch 



Hostile verbs 



Polite, cheerful intonation 



Use of expressive intensifiers 



Use of questions to express 
opinions 



Interruptions in 
conversation with 
females 

Greater amount of 
talking 

Lower pitch levels 
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In two studies by Papalewis & Brown (1989, 1990) of 

35 faculty (17 male and 18 female), with 729 students 

(240 male and 499 female), across 30 different graduate 

and undergraduate classes in Education, the following 

results (see Table 4) were found: In terms of 

communication patterns used by university faculty, as 

predicted from the literature, significant differences were 

found between female and male faculty, such that, 

Females, more than males, were rated 

significantly higher on items regarding: 

*used personal examples in elaborating 
course content, 

*were emotionally involved at times with 
course topics 

*used a wide range of rate and pitch when 
speaking 

*pointed out content areas of personal 
uncertainty 

In terms of student evaluation of instruction, significant 
differences were found between male and female faculty, 
such that: 

Males, more than females, were rated 

' significantly higher on items regarding: 

*CGurse clarity, 
*stating objectives, and 
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^organization 

Females, more than males, were rated 

significantly higher on items regarding: 

^enthusiasm, and 

*inte'i'esi. 

In terms of relationships between gender communication 

patterns and student evaluation of instruction, small to 

moderate positive correlations were found between: 

*the use of personal examples and all evaluation 
items, 

^emotional involvement and items reflecting 
enthusiasm and interest, and 

*the use of wide range of rate and pitch, when 
speaking with items concerning enthusiasm 
and interest. 

An inverse relationship was found betv;aen the pointing out 
of areas of personal uncertainty and items regarding 
course clarity, stating of objectives, and organization. 

Papalewis and Brown (1989, 1990) concluded that 
students do notice gender communication pattern 
differences and such differences are related to specific 
evaluation items. 



Sociocuitural Stereotyping 

As was found with gender, intercultural factors 
affecting interaction and outcomes of conversation based 
on cultural differences requires focused research 
attention. ~- Differences in socialization not only affect how 
others perceive and behave, but also how ones' own 
perception and behavior may contribute to others 
perception and behavior. 

Schnell (1987) defined communication as the 
transmission of information, Ideas, attitudes, or emotions 
from one person to another, by conveying those ideas 
through written or spoken symbols or other verbal or 
nonverbal signs. Perceptions, Schnell (1987) stated, are 
information based on inputs from our senses and 
interpretation of this data on past and - present experiences. 
Hence, verbal and nonverbal symbols influence interracial. 
Interpersonal, and intercultural communication in negative 
and positive ways. Some of the negative effects include 
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racist language and excluding behavior. Prejudice often 
exemplifies itsel? witiiin these negative effects. 

As cited by Booth-Butterfield & Jordan (1989) other 
researchers have identified these race differences: 
African Americans and whites differed in: how they 
reduced uncertainty in initial interactions; the question 
asking pattern and talk timo varied by race and gender; 
African American communicators appear to use eye contact - 
differently than do white communicators; African American 
communicators exhibit more assertive behavior; and, 
African American communicators displayed a heightened 
emotional expressiveness. Hence, African American 
communicator style is characterized as more outgoing, 
assertive, and expressive than white communicator style. 

Foeman and Pressley (1987) found that African 
Americans tend to confront individuals immediately when 
dealing with conflict. As cited by Foeman and Pressley 
(1987), Royce (1982) stated th^.t an African American must 
fairly consistently deal with stereotypes of African 
Americans, and whites, though they tend to think of each 
other as individuals, think of blacks in more sweeping 
terms. 
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Booth-Butterfieid and Jordan (1989) stated that in 
order to elicit favorable evaluations, African American 
communicators often find they need to emulate white 
communication style in educational settings. Citing 
Rubins' study (1986), African American communication 
effectiveness suggested that adaptation to predominantly 
white standards must occur for African Americans to be 
perceived as effective communicators in racially 
heterogeneous classrooms. 

Even less cross-cultural research has been done with 
Hispanics, Native Americans, and Asian Americans. One 
study done by Meredith (1988) found that Japanese 
Americans and female students preferred a more "female" 
style of teaching, where the ' teacher exhibited openness, 
warmth, respect, and accessibility. 

Acknowledging the complexity of contextual variables, 
and the implications of sociocultural variables, should lead 
us in the direction of future research. The sociocultural 
variables present another dimension required to accurately 
assess student evaluation of instruction. The macro 
approach of generalizabiilty of findings in thik research 
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erea, needs to be balanced with a micro approach: untif 
stereotypic dimensions inherent in human behavior are 

explored, we will continue to have research reflect the 
dominant culture. 

Implications 

As stated by Baker (1984) to avoid the trappings of 
ethnocentric bias, researchers must begin to focus on 
cultural values, different verbal and nonverbal 
communication norms, and stereotyping. implications for 
future research are: (1) Teacher evaluation instruments 
in practice should be analyzed for gender and race 
differences in communication patterns; (2) General 
university-wide norms or even faculty-wide norms in a 
particular college/school or multi-program department, 
should not be established; (3) Possibly, the most useful 
evaluation system will include common contextual variable 
norms, and; (4) More research is needed in sociocultural 
contexts to ensure student evaluation of, instruction 
instruments are not stereotypic biased. 
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TABLE 1 DIMENSIONS OF CONTEXT VARIABLES 

COURSE VARIABLES 
Required/elective, Day or evening, Course level, Lecture v 
discussion. Learning value. Organization, Exams/grades, 
Assignments, Workload difficuHy, Class size, and 
Academic.: discipline 

INSTRUCTOR VARIABLES 
Rank, Gender, Full-time v. part-time. Individual rapport, 
Years teaching, and Personality characteristics (Le., 
warmth, enthusiasm) 

STUDENT VARIABLES 
Full-time v. part-time. Expected grade. Prior interest, 
Academic major v. minor. Grade point average. Group 
interaction. Gender, and Personality characteristics 

ADMINISTRATION VARIABLES 
Student anonymity. Direction giving. Instructor remains in 
classroom, When in semester evaluations are given, etc. 

INSTRUMENT VARIABLES 
Placement of items. Number of response alternatives, 
Negative wording of items, and the Labeling of all scale 
points versus labeling only end points. 

18 
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TABLE 2 GENDER PATTERNS OF COMMUNICATION 



Male Patterns 



More frequent use of 
joking 

Shakeshaft,1987, 

Aggressive Speech 
Scott, 1979 



Interruptions in 
conversations with 
females 
Scott, 1979 



Greater amounts of 

talking 

Scott, 1979 



Lower pitch levels 
Scott, 1979 



Demanding voice 
Deep voice 
Boastful speech 
Quantifying modifiers 
Use swear words 
Dominating speech 
Loud speech 
Show anger 
Straight to the point 
Militant speech 
Hostile verbs 
Use slang 

Authoritarian speech 
Forceful speech 
Lean back while talking 
Aggressive speech 
Blunt speech 
Interrupt women 
Humor in speech 
Passive voice 
Prefers impersonal 
Generalizes, 3rd person 
Evidence of fact rather 

than feeling 

Scott, 1979; 
Shakeshaft, 1987 > 
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TABLE 3 GENDER PATTERNS OF COMMUNICATION 



Female Patterns 



Correct speech forms 
Scott, 1979 



Dynamic intonations, 
wide range of pitch 
Shakeshaft, 1987 



Polite, cheerful 
intonation 
Scott, 1979 



Use of expressive 
intensifiers 

Scott, 1979; Lakoff, 1975; 



Use of questions to 
express opinions 
Lakoff, 1975; Shakeshaft, 
1987 



Enunciate clearly 
High pitch 

Use hands and face to express 

ideas 

Gossip 

Concern for listener 

Gentle speech 

Fast speech 

Use of intensifiers 

Talk about trivial topics 

Wide range of rate and pitch 

Friendly speech 

Talk a lot 

Emotional speech 

Use many details 

Smooth speech 

Open, self-reveaiing speech 

Enthusiastic speech 

Smile a lot when talking 

Good grammar 

Polite speech 

Question intonation 

Tag questions 

Use of qualifiers 

Superlatives, diminutives 

Scott, 1979; Shakeshaft, 

1987 
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TABLE 4 



COMMUNICATION PATTERNS AND STUDENT 
EVALUATION OF INSTRUCTION 



Communication Pattern Differences 

Females, more than males, were rated significantly higher on 
items regarding: 

Used personal examples in elaborating course content, 
Were emotionally involved at times with course topics 
1 Used a wide range of rate and pitch when speaking 
Pointed out content areas of personal uncertainty 

Student Evaluation Of Instruction Differences 

Males, more than females, were rated significantly higher on 
items regarding: 

Course clarity, 

j Stating objectives, and 

Organization 

Females, more than males, were rated significantly higher on 
items regardiri: 

Enthusiasm, and 

interest. 

Relationship Between Gender Communication Patterns And Student 
Evaluation Of Instruction small to moderate positive correlations 
were found be^veen: 

The use of personal examples md all evaluation items, 

Hiiiotional involvement and items reflecUng enthusiasm and 
' interest, and 

The use of wide range of rate and pltcH^, when speaklag 

with items concerning enthusiasm and interest. 
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TABLE 5 



AFRICAN AMERICAN PATTERNS 
OF COMMUNICATION 



Confront .individuals immediately 



Foeman & Pressley, 
1987 



Tentative 



Booth-Butterfield & 
Jordan, 1989 



Use Eye Contact 



High Emotioiral Expressiveness 



Assertiveness 



Aiello & Jones, 
1970; Erikson, 
' 1979,- LaFrance & 
Mayo, 1976 

roeman & Pressley, 
1987; Gumperz & 
Tannen, 1979 

. Foeman & Pressley, 
1987 
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